nit: I was trying to figure out where this is set and until I reviewed the script it wasn't clear that is what set it. In other cases we have made that a parameter into the script so things were a little more connected. It looks like you are setting 2 variables which is probably why you opted not to do that.

Previously they were passing this into the python script and calling it prior to every sparse checkout that wasn't the default. There's no real point in passing in an environment variable to set if it's always being set in multiple places. Further, I'd added a second variable for ServiceDirectires which isn't yet being used but will be so I saw no point in passing in variables that were being expected.

Did you intent to move the PreBuildSteps after the sparse checkout?

This was intentional, otherwise there's a chicken or egg problem with the pre build scripts not being checked out with the initial sparse checkout pass.

You should consider using Get-java-PackageInfoFromRepo from language-settings.ps1 to try and start using a common function for gathering properties for the projects.

@weshaggard, we'd have a chicken and egg there. That function requires the service directory. The common functions, for Java anyways, are far less common than one would think. Things under common aren't really common outside of the fact that they're specifically being used for a targeted piece of processing (like docs updating or .MD file verifications) and are mostly not useful outside of the context in which they were initially written.

I believe that function should work without a servicedirectory as well. Mostly I was wanting to try and have a common function we can use to parse and produce the properties.

I'm not sure what the limit is but I suspect there is a limit on the size of a DevOps variable or a Env variable. So might be something we need to think about.

@weshaggard - Previously the sparse checkout was granular down to the artifact's directory in some cases there were 120+ items added to the sparse checkout (like core's FromSource run for example). I changed things to just simply do the service directory which turned 120+ into like 30. If there's a limit it would have been hit long before I'd made my changes which dramatically trimmed things down.

I can't find the docs for it right now, but I remember the limit being large enough that it hasn't been a concern thus far.

Is there any good reason to do this in both scripts as opposed to just running both scripts?

The scripts generate dramatically different lists for the same list of artifacts. The generate_from_source_pom produces a list of sparse checkout directories that's necessary for the FromSource run which includes the service directory for the list and every service directory that has a dependency on any project in the project list. It also generates the ClientFromSourcePom.xml which is only needed in two places. Generate-ServiceDirectories-From-Project-List.ps1 only generates the list of service directories for the project list and nothing more.